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ABSTRACT 



This paper addresses issues in assessment of college 
students, outcome studies, and attrition research in the context of trends 
toward requiring greater accountability from institutions of higher 
education. First the paper considers how assessment, student outcomes 
studies, and attrition research terminology might be more clearly defined. It 
suggests that these three areas of inquiry, although distinctive, are highly 
interrelated phenomena. The paper then attempts to operationalize definitions 
and outline a recursive research model appropriate for research in these 
areas. Finally, it briefly describes how one small liberal arts university is 
currently thinking through the process of implementing a longitudinal 
assessment program including student outcomes studies and student attrition 
research through a recursive research model and operationalized definitions. 
Stressed is the need for institutions to plan their research to be consistent 
with the educational goals of the institution. (Contains 17 references.) 
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"ASSESSMENT, OUTCOMES MEASUREMENT AND ATTRITION" 
{Reflections, Definitions and Delineations} 

One need not immerse oneself in the literature on college 
student assessment, outcomes studies and attrition research to 
any great degree to realize that these areas of inquiry are 
still very much in the process of being defined as legitimate 
avenues of educational research. Attrition research began in 
earnest about a quarter of a century ago, Astin (1975), Cope 
and Hannah (1975), Tinto (1975), Pascarella and Terenzini 
(1978), Johnson (1980), while efforts to develop models for 
assessment and student outcomes studies are more recent, Astin 
(1991 & 1993), Light (1992), Angelo and Cross (1993), Banta 
and Associates (1993), Ewell (1985 and 1988), and many others. 
Despite this volume of literature, there is still a good deal 
of ambiguity in the way the terms assessment, outcomes 
measurement, and attrition are defined for research purposes. 
For example, the National Association of Student Personnel 
Administrators (1995), recently published a booklet titled 
"Successful Student Outcomes Assessment." In this short work, 
NASPA included a partial inventory of available instruments 
currently in use for assessment and outcomes research. The 
title of NASPA' s publication implies that the terms assessment 
and outcomes are, if not interchangeable, then at least 
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similar enough so as to require no further delineation 
regarding the precise meaning of these terms. This notion is 
further reinforced when one examines carefully the inventory 
of assessment/outcomes instruments NASPA selected for its 
partial inventory. That list included more than 27 
instruments which addressed topics as diverse as scholastic 
aptitude, vocational interests, career paths, concepts of 
work, capacity to realize opportunities, extra-curricular 
.activities, critical thinking, judgement, confidence, sexual 
identity and a host of other subjects. The diversity of 
subject matter which these inventories purport to measure 
implies that regardless of its emphasis, all student research 
is, in effect, defacto assessment research. And yet not all 
researchers would agree. For example, Mentkowski (1994) 
recently observed that educators must somehow "connect 
assessment with teaching and learning." She views the 
assessment process as an altogether different exercise from 
outcomes research. Her view, which is shared by others, is 
that outcomes studies strictly speaking should address 
academic and intellectual growth, an emphasis manifestly 
different from assessment which is often used to imply data 
gathering . 

Despite the ambiguity in how these research areas are 
presently defined, there is little doubt that educational 
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administrators recognize the value of the assessment process 
largely because this research enables them to better 
understand the students they seek to educate. They also 
understand that data gathered under the rubric of student 
assessment and outcomes studies have the potential to mitigate 
student attrition by anticipating which students a-priori have 
a demonstrable likelihood of failing to complete an academic 
program. In a larger context, when institutions establish 
carefully defined assessment programs, they are in effect 
demonstrating their interest in learning as much as they 
possibly can about their students and their students' needs. 
This in turn reassures legislatures which allocate funds in 
support of public higher education, governing boards which 
approve budgets for private institutions, and an increasingly 
skeptical public that institutions of higher learning are in 
fact trying to maximize the likelihood of graduation for each 
student. At a time when an investment in higher education is 
so closely correlated in the public's mind with the value of 
higher education, institutions are seeking to do everything 
possible to maintain and increase graduation rates thereby 
justifying the very considerable investment which the cost of 
a college education represents. 

Toward that end, the purpose of this reflection is three- 
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fold. First, observations are offered on how assessment, 
student outcomes studies and attrition research terminology 
might be more clearly defined and made more useful to 
consumers of this research. Second, the suggestion is made 
that these three areas of inquiry, each distinctive in and of 
itself, are nonetheless, highly interrelated phenomena. 
Finally, in operationalyzing definitions and outlining a 
recursive research model, this paper highlights how one small 
liberal arts university is currently thinking through the 
process of implementing a longitudinal assessment program 
including student outcomes studies and student attrition 
research. 

When administrators plan assessment programs, they must 
first distill the term assessment into its component parts. 
Broadly defined, assessment consists of three highly 
interrelated phenomena each of which has its own distinctive 
characteristics. The first of these components is assessment 
itself. According to the (1995) New American Heritage 
Dictionary of the English Language, the word assessment means 
to determine the value of something, or to determine its 
significance or its extent. In higher education, this implies 
data gathering; specifically, gathering various kinds of 
descriptive data on students who comprise the institution's 
clientele. The term assessment therefore should be carefully 
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defined to mean the collection or assembly of data, 
demographic, vocational, personal, for example. To employ 
the term in this way enables researchers to differentiate 
assessment from other measurement activities. These data can 
then be used in a variety of ways, perhaps none more important 
than assisting institutions in learning who their students are 
as well as what their students' needs are. 

The first stage of the assessment exercise is to 
establish a baseline of data. These data sets should include 
information on 1) incoming freshmen, 2) sophomores and juniors 
who can be operationally defined as continuing students, 3) 
graduating seniors, 4) graduate students, 5) alumni whose 
graduation date was within 5 years of the date of the survey 
and finally, 6) alumni whose date of graduation was older than 
five years. To accomplish this phase of the research, one 
approach, (there are many others) is to use the National 
Center of Higher Education Management Systems (NCHEMS ) Student 
Opinion and Information Survey (SOIS) as a survey vehicle. 
These instruments are specifically designed to collect 
information, not to measure academic outcomes . In addition, 
these surveys are easy to administer, clearly understood by 
the students who complete them and easily tabulated for 
analysis. "NCHEMS" survey instruments enables the researcher 
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other things, students' experiences, expectations, 
aspirations, specific goals, and their evaluation of the 
institution and its services. In addition to telling 
institutions just who their students are, these surveys also 
summarize additional information having to do with the overall 
quality of the association between the students and the 
institution, something all academic and student affairs 
administrators want to know. In addition, these group 
profiles can also serve as a valuable foundation upon which 
strategic planning might be based. In short, the initial 
phase of the assessment process should be a relatively easy 
exercise of collecting extensive descriptive data on the 
institution's audiences. All other things being equal, it 
seems a perfectly reasonable way to begin the assessment 
process. In defining assessment in this way, institutions are 
able to differentiate this activity from student outcomes 
measurement and attrition research which constitutes separate 
but obviously related, collateral activities. 

To begin the second phase of research, institutions 
should identify a group of students on whom baseline data 
reflecting rudimentary academic skills can be established. 
Those students should be asked to complete a set of 
examinations which measures verbal and quantitative abilities. 
The collection of such information is of course commonplace at 
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many institutions. As elsewhere, these baseline data enhance 
academic advising and program planning. But these data have 
an additional purpose. Students who participated in this 
phase of the study can then be tracked to determine whether or 
not they acquired the skills in English and mathematics that 
institutions seek to instill in their students. Pre and post 
designs are particularly appropriate for this purpose. During 
their first two years, these students then take a variety of 
typical freshman and sophomore English and mathematics 
courses. Pre and post test analysis, can determine whether a 
statistically significant difference could be identified 
between students 1 initial scores and their subsequent scores 
on alternate versions of, for example, the ASSET examination 
published by the American College Testing Service. If 
statistically significant differences for these dependent 
groups are found, institutions may conclude that the 
intervening course work did, in part, contribute to those 
differences. Such results however speak only to elementary 
E n 9lish and mathematics skills, not to the broader issue of 
higher order, relational and abstract thinking. As long as 
the variables used in outcomes measurement are very narrowly 
defined and easily quantified, it is not a particularly 
difficult matter to test and re-test groups of students to 
determine whether or not academic objectives are being met. 
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Scores on standardized math and English tests are a case in 
point. More substantial outcomes measures however, 

specifically those which test for the acquisition of critical 
thinking skills and complex problem solving, are much more 
difficult to identify. In order to determine whether or not 
certain higher order academic skills have been acquired by 
students, institutions must first determine exactly what they 
are attempting to impart to students through the process of 
education. Most faculty would agree that basic skills in 
English and math are desirable outcomes of the education 
process. But beyond this basic agreement, faculty are often 
quite polarized regarding their opinions on what characterizes 
an educated person. Variations of the often repeated themes 
of thinking with clarity, achieving depth and breadth in a 
field of knowledge, acquiring the capacity to grapple with 
moral and ethical problems, distilling complex problems into 
their component parts or learning to appreciate the process 
whereby knowledge itself is acquired continue to be discussed 
as desirable traits for educated persons to possess. And 
herein lies one of the central problems with academic outcomes 
studies. Not only must each institution decide what it wishes 
to impart to its students, it must also devise ways to 
determine whether those objectives have been met. 
widely accepted truism that the more sophisticated 
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intellectual behavior becomes, the more difficult it is to 
quantify and measure. It is equally true that the primary 
reason higher order reasoning skills are often not part of 
student outcomes research is that university faculties simply 
cannot agree on what should be taught and how such skills, 
whatever they are perceived to be, should be measured. In 
discussions at the University of Great Falls which focused on 
how one could determine whether or not graduates were 
reflections of what faculty wanted them to be, it became clear 
that our perceptions of what constituted an educated person 
were quite disparate. We also agreed that if left unaltered, 
our various notions of what an educated person should be would 
preclude any long term meaningful student outcomes research. 
If we could not even agree on what belongs in the curriculum, 
how then could we begin to think through an outcomes process? 
Responding to this impasse, we began a two-year long dialogue 
on reconfiguring the core components of our undergraduate 
curr iculum to bring about some common agreement on what 
comprised a relevant and rigorous course of undergraduate 
study. Only by establishing common goals were we able to 
reach some consensus on what our graduates should know. And 
only then could we undertake a meaningful dialogue on how to 
begin to structure measurement instruments to tell us if the 
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goals we had set for students were being realized. Student 
outcomes research therefore has the potential to tell an 
institution not only a great deal about its students ' 
intellectual achievements but also a great deal about the 
faculty who educate them. This then, is the real value of 
student outcomes research and what sets it apart from simple 
assessment. Outcomes measurement which gauges real 
intellectual depth and breadth is only possible when faculty 
share a common vision regarding what intellectual 
characteristics are desirable and how those skills might be 
measured appropriately. 

The third component of the assessment triptych addresses 
student attrition research, the goal of which is two-fold. 
The first task is to develop predictor equations which 
identify students who fit a "high risk" profile. The second 
goal is dictated by the results of the first. It entails the 
design and implementation of programs to decrease the 
likelihood that a particular student will become an attrition 
statistic. How is the first goal to be accomplished? Each 
institution should develop its own set of predictor equations 
using assessment and outcomes data on students who have 
graduated compared to those who have not. Multivariate 
statistics, especially multiple regression and discriminant 
analysis, are praticularly useful in this area of research. 
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Once predictor models have been developed, assessment and 
outcomes data can then be used to predict into which category, 
(persisters, voluntary withdrawers, or academic dismissals) 
any given student's profile suggests he or she is likely to 
fall. Demographic variables, pre-matriculation academic 
variables and other measures can be used to isolate the best 
predictors of subsequent academic standing. Once students are 
identified as high risk, institutions can intervene with 
tutorial programs, counseling, remedial work and the like to 
attempt to mitigate what initially might seem to be an 
inevitable outcome. 

In summary, the three activities commonly referred to as 
assessment, outcomes measurement and attrition prediction are 
in reality separate but highly interrelated phenomena. None 
of the three can be meaningfully understood in isolation or 
without reference to the other two. As administrators and 
researchers develop and continue to develop assessment 
programs, it will be helpful to differentiate these research 
activities into three specific areas so that each component is 
clearly defined and its relationship to the other two can be 
made clear. The assessment process as defined here should be 
the initial exercise. Periodic outcomes measurement can then 
be used in conjunction with assessment data to develop and 
refine predictor equations which can identify at risk 
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students. However, in order for the process to serve the 
institution and its students, institutions must plan their 
research so that it is consistent with the overall educational 
goals as determined by its faculty. Outcomes measurement 
research methodology must be drawn directly from those goals 
and aspirations. Models for assessment, for outcomes 
measurement and for attrition prediction must be developed by 
individual campuses for localized use. These models must be 
constructed using clearly defined, operational definitions of 
assessment and related terms. Failures to provide such 
definitions almost certainly guarantees continued obfuscation 
and ambiguity in this area of research which is so important 
to institutions of higher education. 
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